The testing of software is normally the phase of software development which is assigned the fewest resources and given the shortest amount of time to accomplish its task. These facts translate to software products having poor quality and exhibiting low customer satisfaction. A major task of the software tester is to create test cases that exercise a software product and determine whether the software is able or unable to provide expected results. In the past, the generation of test cases was entirely manual, very complicated, and many times did not exhaust all of the test cases that were needed to be run. More recently, software testers have been embodied into semi-automatic and automatic test systems.
One such software system accepts as inputs, test cases expressed in the Backus-Naur form (a notation that specifies a programming language syntax) and automatically generates text files therefrom. Because the Backus-Naur form is a pseudo-like language and is very cryptic, users find it difficult to use. In U.S. Pat. No. 4,819,233 to Delucia et al., a test system is implemented by generating a test driver program which is compiled and linked to a source code listing to be tested. The source code is instrumented to generate, when the code is executed, an indication for each block of code that the statements of that block have been executed. Through a series of tests, all blocks of code are executed using a series of test cases. The Delucia et al system is a further example of the use of an input language to achieve the generation of test cases.
By contrast, U.S. Pat. No. 5,067,129 to Evans et al. and 5,187,788 to Marmelstein both employ graphical interface systems to enable generation of software packages. Marmelstein creates a graphical representation of a program, which representation is then directly converted to an operating code. The Marmelstein system provides three different, consistent views of the same program. With the first view, the programmer lays out the initial program package specifications. A second view allows the programmer to create and manipulate data structures and to define local variables. The last view enables the programmer to define the control flow of subprograms.
Evans et al employ a graphical interface to automatically generate test cases for the testing of a service processor associated with a computer. The Evans et al. system includes a knowledge base that stores a map indicating a correct arrangement of screens, which screens are stored by the computer. A program generator is coupled to the user interface and the knowledge base and identifies from the computer's memory, screens in a group. The program generation further directs the service processor to call up screens of the group and to test the pathing between the screens by comparing expectations of test programs to the screens actually called up.
Evans et al. employ a classical tree structure to define the test cases. Such a tree structure has the following attributes; nodes-inherit attributes from their parents; all nodes have the same attribute formats; nodes cannot have multiple parents; nodes can have multiple children; and a tree must have at least one root node. In such a tree structure, a node is an "object" containing data and is generally represented by a polygon in the diagram. A "link" is a defined relationship among nodes and is represented by a line with an arrow. The arrow points to the child of the relationship. A root node is a node which does not have a parent.
While pure tree structures can graphically show test cases, they exhibit a number of shortcomings. First, they are unable to handle probabilities which define the likelihood of a particular branch of the tree being employed in a test case. Second, a pure tree can only generate one combination of circumstances and becomes extremely complex when required to illustrate plural combinations of test cases.
A further known method for the generation of test cases is through the use of cause-effect graphs. (See "Art of Software Testing", G. Myers, John Wiley & Sons (1979) Such graphs enable combinations of input conditions to be derived for test cases. The testing of combinations of input conditions is not simple since the number of combinations is usually very high. Cause-effect graphs employ nodes and links and, further employ logical operators such as AND, OR, and NOT. To derive a cause-effect graph, the software specification is divided into "workable" pieces because otherwise, a cause-effect graph becomes unwieldy. Causes and effects are then identified, with a cause being a distinct input condition or a class of input conditions and an effect being an output condition or a system transformation. The semantic context of the specification is then analyzed and transformed into a Boolean graph that links the causes and effects (i.e. the cause-effect graph). The graph is then annotated with constraints describing the combination of causes and/or effects that are impossible. The graph can then be utilized to generate the software test cases.
Cause-effect graphs have different attributes than classical tree structures. Nodes do not inherit from their parents; there are different types of nodes with different types of attributes; a root node does not have to exist; and nodes can have multiple parents and multiple children. As a result, cause-effect graphs can quickly become extremely complex, difficult to comprehend, and unwieldy to implement on a graphical user interface. Additionally, the logical operators that are specified neither accommodate probabilities nor enable loops.
Accordingly, it is an object of this invention to provide an improved system for generating test cases for software cases.
It is another object of this invention to provide an automated system for generating test cases which can be implemented with a graphical user interface.
It is yet another object of this invention to provide an improved system for generating software test cases which can accommodate probability functions and multiple repetitions of test cases without creating unduly complex test case representations.